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Abstract 

The  report  contains  a  brief  summary  of  aspects  of  the  Maximus 
reliability  point  and  interval  estimation  technique  as  it  has  been  applied  to 
the  reliability  of  a  device  whose  surveillance  tests  contain  a  succession  of 
stressful  environments.  The  lower  confidence  limits  appear  to  be  ultra 
conservative.  It  is  argued  that  the  serial  system  approach  to  modeling  of 
the  cumulative  effect  of  the  stresses  is  inappropriate.  Some  alternative 
modeling  is  suggested.  Some  numerical  comparisons  are  offered. 

1.  Introduction. 

The  Naval  Surface  Warfare  Center  in  Crane,  Indiana  (NSWC  -  Crane)  is  engaged  in  a 
number  of  reliability  assessment  programs  for  ammunition  and  pyrotechnic  devices.  Often 
reliability  figures  must  be  developed  from  information  derived  from  diverse  sources.  It  is 
common  to  “bench”  test  the  components  and  subsystems  individually  in  order  to 
understand  the  sources  of  unreliability.  Such  tests  can  be  of  the  binary  type,  success  or 
failure,  and  the  component  reliability  mformation  is  integrated  into  a  computation  of  the 
system  reliability. 

Computational  methodologies  have  been  developed  to  support  these  activities  [2,3,6, 
7,9,14].  Certain  aspects  are  of  immediate  interest.  Complicated  systems  are  decomposed. 


UnG  QUAMTl'  mSPEUTTIB 


Those  collections  of  components  that  utilize  their  units  in  either  a  purely  series  or  purely 
parallel  way  are  identified  as  subsystems.  Their  reliabilities  are  estimated  from  the 
component  test  data  by  cither  forming  products  of  success  rates  (series  case),  or 
complementing  the  products  of  failure  rates  (parallel  case).  TTiese  subsystems  are  then 
viewed  as  pseudo  components  whose  performances  are  treated  as  Bernoulli  random 
variables  with  success  rate  estimated  as  just  stated.  Since  a  niunber  of  component  tests 
were  performed  to  produce  these  rates,  not  all  having  the  same  sample  numbers,  rules 
[8,9,14]  have  been  developed  to  convert  the  collection  of  component  sample  numbers  into 
a  single  number  which  is  used  to  represent  the  number  of  times  that  the  pseudo  subsystem 
has  been  tested.  Further,  this  number  is  then  multiplied  by  the  estimated  reliability  in  order 
to  obtain  an  ‘equivalent  number  of  successful  tests’.  Of  course  this  latter  number  need  not 
be  an  integer,  but  stiU  it  is  regarded  as  being  a  binomial  random  variable  in  a  sense  to  be 
described. 

The  technique  cascades.  Several  such  pseudo  components  combined  in  either  purely 
series  or  purely  parallel  ways  are  given  like  treatment;  sample  sizes  and  success  counts  of 
these  new  pseudo  components  are  computed  as  before.  Neither  of  these  two  quantities 
need  be  integral.  Nonetheless,  it  is  convenient  to  speak  of  them  as  sample  sizes  and 
binomial  random  variables.  Only  the  ratio  is  required  to  produce  a  success  rate,  and  the 
Incomplete  Beta  function  can  be  used  to  compute  confidence  intervals.  Thus,  neither  the 
sample  ‘size’  nor  the  ‘number’  of  success  need  be  integers.  Lower  confidence  bounds  Gcb) 
of  the  ultimate  system  reliability  are  of  particular  interest.  Generally  the  Icb’s  are 
conservative. 

Ideas  similar  to  those  just  described  are  also  being  applied  to  treat  the  ‘lot  acceptance’ 
and  ‘shelf  life’  problems;  that  is,  the  problem  of  the  initial  acceptance  of  a  fieshly 
produced  lot  and,  at  later  times,  the  (periodic)  surveillance  testing  of  the  accepted  lots  to 
determine  if  time,  treatment  and  the  environment  have  affected  performance.  Lots  that 
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may  have  deteriorated  to  below  acceptable  levels  need  to  be  identified  and  discarded.  To 
this  end  the  test  plans  contain  provision  for  subjecting  the  items  to  sets  of  specified 
treatments  designed  to  provide  stress  and  hasten  the  exposure  of  defects.  The  test  plans 
specify  the  partitioning  of  the  selected  sample  into  subsets  such  that  the  items  in  a  single 
subset  receive  one  and  only  one  of  the  stresses.  No  stress  at  all  is  usually  one  of  the 
subsets.  Testing  ultimately  results  in  the  destruction  of  the  items. 

Although  no  single  item  is  subjected  to  mote  than  one  of  the  stresses,  there  is  a 
requirement  that  the  reliability  estimates  refer  to  items  that  have  received  all  of  the 
stresses.  This  requirement  is  approached  with  the  same  methodology  indicated  in 
paragraphs  two  and  three  above,  i.e.  treat  the  various  stresses  as  components  of  the 
system.  In  this  case  they  are  treated  as  independent  components  of  a  serial  system.  The 
Icb’s  produced  when  so  doing  appear  to  be  overly  conservative. 

The  present  report  reviews  this  situation  as  it  pertains  to  the  marine  smoke  and 
illumination  signal,  MK  124  MOD  0,  and  presents  some  alternatives  that  may  yield  more 
realistic  estimates.  Section  2  contains  a  description  of  the  performance  characteristics  of 
the  device  and  the  nature  of  the  test  plan.  The  mathematical  modeling  and  notation  are 
presented  in  Section  3.  Section  4  contains  summaries  of  six  methods  for  treating  the 
reliability  calculation;  two  from  the  Sandia  approach  [2,14],  two  from  the  L.J.  Gleser  [6] 
approach,  and  two  by  the  author.  Next,  the  six  methods  are  subjected  to  numerical 
comparison  using  two  test  cases  in  Section  5.  This  section  also  contains  some  single 
modeling  that  could  lead  to  improvement  of  the  Icb’s,  but  at  the  cost  of  heavier 
computation.  Such  computations  have  not  been  pursued.  Finally  some  log  linear  modeling 
is  proposed  in  Section  6.  This  is  done  to  show  how  one  could  exploit  more  of  the 
information  that  is  contained  in  the  test  results  and  to  suggest  ways  to  set  decision  rales  in 
anticipation  of  changes  in  the  testing  requirements.  It  is  expected  that  the  changes  will  not 
permit  any  failures  at  all  in  order  to  accept  a  lot.  A  summary  section  follows  and  contains 
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commentary  on  the  appropriateness  of  the  methods.  Several  appendices  are  attached  that 
document  a  variety  of  details  that  appear  in  support  of  the  report 

2.  Description  of  Item  Characteristics  and  Test  Plan. 

The  device  is  ^proximately  5.408  inches  long  and  1.7  inches  in  diameter.  It  weighs 
about  half  a  pound.  A  flare  candle  is  at  one  end  and  a  smoke  candle  is  at  the  other.  There 
are  five  performance  characteristics  to  be  met  by  the  signal: 

a.  Display  color:  Exhibit  orange  smoke  from  one  end  and  a  red  flare  from  the 

other. 

b.  Function:  Ignite  and  produce  the  displays. 

c.  Delay:  No  more  than  three  seconds  from  initiation  to  the  generation 

of  display. 

d.  Display  Times:  The  display  time  begins  after  the  delay  time  ends;  minimal  and 

maximal  display  times  are  specified  for  each  of  the  two  types 

of  signal. 

e.  Safety:  The  igniter  will  not  separate  from  the  container  during 

function  (b). 

These  are  grouped  and  an  item  test  can  have  one  of  four  outcomes.  That  is,  a  test 
failure  will  be  marked  as  being  in  exactly  one  of  the  first  three  outcomes;  the  fourth  is  for  a 
successful  test.  These  outcomes  are: 

a«&e.  Color  display  or  safety  failure. 

b.  Ignition  failure. 

c&d.  Delay  or  burning  time  specification  failure. 

sue.  Success;  no  functioning  out  of  zone. 

Prior  to  testing,  an  item  is  subjected  to  one  of  the  following  treatments  whose  details 
are  specified  in  [12].  It  must  withstand  them  without  exploding  or  burning. 

i.  Five  foot  drop. 

ii.  Forty  foot  drop. 

iii.  Vibration. 

iv.  Temperature  and  Humidity. 

V.  High  Temperature. 

vL  Low  temperature. 

vii.  No  stress. 
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The  system  test  procedures  include  visual  inspection  and  X-ray  prior  to  proceeding 
with  the  ultimate  performance  tests  which  are  destructive  in  nature.  Of  course  the 
inspections  are  non  destructive  and  when  they  identify  defects  in  the  samples  the  lots  are 
rejected  out  of  hand.  No  further  testing  is  done.  No  reUabilily  computations  are  required. 
For  those  lots  that  pass  the  non  destructive  phase,  there  is  the  requirement  that  reliability 
figures  should  reflect  the  item’s  ability  to  perform  after  having  been  subjected  to  aU  of  the 
stressful  situations  prescribed. 

A  random  sample  of  N  items  is  selected  from  the  lot.  They  are  partitioned  into  k 
groups  of  size  «,•  for  i  =  1,...,  k.  Thus  each  sample  unit  is  subjected  to  exactly  one  of  the 
treatments.  Further,  the  test  plan  specifies  the  threshold  for  the  allowable  number  of 
failures  in  each  treatment  group  and  by  failure  type.  The  common  way  is  to  provide  a  pair 
of  values  separated  by  a  ‘slash’,  e.g.  1/2,  which  means  that  a  successful  test  in  an  outcome 
type  is  allowed  to  suffer  zero  or  one  failures  among  its  samples,  but  must  be  designated 
as  an  item  failure  if  it  fails  in  two  or  more  cases.  An  excerpt  of  a  test  plan  sheet  is 
exhibited  in  Appendix  A.  For  this  plan  k  =  l.  The  present  description  has  been  curtailed  to 
show  only  those  aspects  that  may  be  of  immediate  use  in  the  reliability  computations.  Full 
details  can  be  found  in  [12]. 

3.  The  Reliability  Model. 

The  goal  is  to  produce  point  estimates  and  lower  confidence  bounds  for  the  reliability 
of  the  device,  in  this  case  the  marine  signal.  The  definition  of  reliability  used  is  that  the 
item  must  perform  acceptably  having  been  subjected  to  all  of  the  specified  stresses. 

The  reliability  computations  presently  in  use  [6,14]  reflect  models  that  assume  a  series 
system  with  independent  components.  Let  us  examine  two  of  these  models  in  order  to 
understand  what  they  reflect  Much  notation  and  jargon  will  be  borrowed  from  Gleser  [6]. 
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Suppose  there  are  ^-1  stress  treatments  and  an  additional  ‘no-stress’  condition 
bringing  the  total  to  k.  This  latter  situation  is  tested  directly.  Gleser  would  call  this  the 
hazard  of  being  produced,  and  it  is  marked  first  in  the  notation.  That  is,  Ei  is  the  event 
that  a  freshly  manufactured  device  performs  acceptably.  Let  £/  be  the  event  that  the  device 
survives  the  stress  environment  for  i  =  2,...,  k.  The  reliability,  R,  as  defined  above  is  the 
probability  of  the  intersection  of  the  events  £i,...,  Ej^.  When  a  single  stress  (say  the  is 
a  treatment  prior  to  a  device  test,  the  success  probability  is  the  probability  of  and  Ej. 
Thus  the  reliability  goal  can  be  represented  as 

R  =  Prl^i  r\E2<~^. 

=  Pr{£i}  Pr{£2  n  £30. .  .nEj,}.  (1) 

When  one  assumes  the  independence  of  the  survivability  of  stresses,  then  one  can  express 
the  reliability  in  terms  of  estimable  quantities 

k 

R  =  Fi{Ei]YlP{Ei\Ei} 

2 

=  Pr{£i  n  £2}.„P{Ei  n  B*}/[Pr{£i}]‘“^ .  (2) 

The  quantities  Ri  =  Pr{£i}  and  Rj  =  Pr{£i  and  Ej)  are  directly  estimable  from  the  test 
using  the  proportion  of  successes.  The  interpretation  depends  upon  what  is  meant  by 
‘surviving  an  environment’.  In  the  large,  it  means  acceptable  performance  after  having 
been  treated  with  the  stress.  If  the  conditional  probabilities  are  not  unity  then  some 
deterioration  has  taken  place.  We  know  not  how  to  measure  the  deterioration  nor  how 
much  of  it  is  acceptable.  The  assumption  of  independence  is  made  as  an  expedient  since  no 
modeling  of  this  deterioration  has  been  proposed.  ITiere  is  a  latent  “coherence” 
assumption  that  says  that  application  of  the  stresses  do  not  improve  the  reliability  of  the 
device. 
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4.  Methods. 

Three  pairs  of  methods  are  outlined  in  this  section;  Sandia,  Gleser,  and  NFS.  The  last 
is  proposed  by  the  author.  The  first  two  have  been  proposed  in  earlier  times.  It  seems 
useful  to  summarize  them  all  at  once  and  in  a  common  notation.  Also,  there  are  two 
versions  of  each  (denoted  I  and  H). 


Sandia:  Let  qy,  q2, ...,  be  the  failure  probabilities  of  the  devices  after  subjection  to  the 
various  stresses.  That  is, 


qi  =  PrjEf  I  for  i  =  1,2, . . . ,  ^. 

Then 

Ri  =  Pr{£i  n  £;}  =  1  -  Pr{£f  u  Ef] 

=  l-^l-^,+Pr{£f  nEf}. 

(3) 

Assume  the  last  term  is  negligible 

Pr{£f  n  £/]  =  0 ,  for  all  i  =  2, . . .,  L 

(4) 

The  independence  assumption  allows  us  to  write 


=Pr{^l  n£3}...Pr{£i 

=  -(Ik) 

(5) 

2 

On  the  other  hand 

R  =  Pr{£i  nE2r\...nE/^] 

=  1  - Prj^i^  kjE^'U.. xjEI I 

1  ij 

k 

1 
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Combine  with  (5)  and  get 


k 

+  (6) 

2 

Suppose  n,  is  the  number  of  items  subjected  to  the  stress  {i  =  I,  ...,k)  and  Zj  is  the 
corresponding  failure  count.  We  use  the  estimators  =  1  —  X/ny.  The  approximation  (6) 

suggests  the  estimator 

R  =  Rp  +  {k-2)qi  (7) 

k 

where  Rp  is  an  estimator  for  the  product  JJ  R^ . 

2 

It  remains  to  get  the  lower  confidence  bound;  two  methods  have  been  put  forth: 
original  [14]  and  the  Gleser  [6]  modification.  We  wUl  call  them  I  and  n.  Both  are  based 
upon  the  use  of  an  equivalent  sample  size  computed  from 

n*  =r[1-R)/v^r)  (8) 

and  an  equivalent  number  of  successes 


x*=nR.  (9) 

Then  the  (1  -  a)  level  lower  confidence  bound  is  found  by  solving  the  equation 

a  =  IB(lcb,x*,l+«* -Jf*)  (10) 

where  IB  is  the  incomplete  Beta  function;  see  Appendix  B.  The  difference  in  the  two 

A. 

methods  lies  in  the  manner  of  estimating  the  variance  of  i?  in  the  denominator  of  (8). 

I.  Use  the  independence  of  the  two  terms  of  (7)  in  the  approximation 


var 


1  2 


+  —  2) 


2  ^l(l~|l) 


min(nj)  nj 

The  use  of  the  mm(n,)  comes  from  the  Lindstrom-Madden  method  [8,14]. 


(11) 
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n.  Gleser  has  suggested  the  use  of  an  approximation  formula  for  the  variance  of  the 
product  of  independent  proportions,  see  Appendix  C.  The  variance  of  Rp,  (the  first  term 

in  (1 1))  is  approximated  by  eq.  (C.l).  The  estimated  form  is 

k  k 


var 


(  k  \ 

HA 

V  2  ; 


k 

riA 

.  2 


iy=2  2 
i^j 


(12) 


These  two  methods  are  called  Sandia  I  and  Sandia  H. 


The  Gleser  Approach, 

Gleser  [6]  suggested  more  exploitation  of  the  independence  assumption  that  is  implicit 
in  the  Sandia  analysis.  Specifically  use  estimators  in  equation  (2),  i.e. 


(13) 

The  lower  confidence  bound  uses  the  Sandia  method,  but  the  denominator  of  (8)  is 
approximated  using  the  delta  method  for  the  ratio  (13)  of  two  independent  random 
variables.  This  is  outlined  in  Appendix  D  and  appears  as  formula  (D.3).  The  approximate 
variance  of  (13)  is 


I. 


Var(^)  = 


Var(fip) 


Ik 


^+{k-2rR^p 


2„2  Var(l-gl) 


2k-2 


(14) 


A 

and  the  point  estimators  Rj  and  qi  can  be  inserted  for  computation.  Then  apply  (8,  9, 10). 
This  is  Gleser  I. 

Gleser  further  draws  attention  to  the  fact  that  R  is  the  maximum  likelihood  estimator 
for  R  under  the  model  that  Xj  ~  Bin(nj-,  p,)  for  /  =  1 , . . . ,  A:  and  independent. 

For  Gleser  n  the  following  crude  ad  hoc  method  is  presented  to  improve  the  lower 
confidence  bound.  The  point  estimate  is  the  same,  (13).  For  an  approximate  lower 
confidence  bound  we  exploit  the  fact  that  the  numerator  and  denominator  of  (13)  are 
independent  random  variables.  These  factors  are  direct  point  estimates  of  Rp  and 
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(1  -  respectively,  and  the  ratio  of  these  latter  two  parameters  represent  R.  We  can 

easily  develop  an  upper  confidence  bound  for  (1  -  qi)  and  a  lower  confidence  bound  for 
Rp.  The  ratio  of  these  bounds,  properly  adjusted,  can  provide  a  lower  confidence  bound 
for  R.  The  implementation  proceeds  as  follows.  Let 

{lcb(^p)</?p}  and  {(l-^i)< ucb(l-^l)} 

each  be  events  of  probability  Vl-a.  Because  of  the  independence,  their  joint  occurrence 
has  probability  1  -  a.  It  follows  that 

n.  Pr{lcb(i^ )/[ucb(l  -  qi )]^"^  <  i?}  >  1  -  cr  (15) 

and  hence  that  the  ratio  in  the  above  event,  denoted  lcb(^),  is  a  lower  confidence  bound 

of  probability  at  least  1  -  a.  Gleser’s  numerical  example  shows  that  this  bound  can  be  a 
noticeable  improvement  over  the  earlier  ones.  See  Table  2. 

NFS. 

This  method  was  motivated  by  the  fact  that  no  item  is  subjected  to  more  than  one 
stress.  So,  the  requirement  of  the  device  being  able  to  function  after  being  subjected  to  all 
of  the  stresses  is  not  reflected  in  the  testing  program.  ITie  model  adopted  to  use  test 
results  for  this  requirement  is  a  highly  conservative  one.  It  says,  in  effect,  that  the  device 
failure  rate  is  the  sum  of  the  failure  rates  associated  with  the  failure  rates  of  items 
subjected  to  the  single  stresses.  This  form  of  accumulation  is  simple  but  overly 
conservative.  Other  models  of  damage  accumulation  can  be  rather  complicated. 

The  author  suggests  that  this  requirement  be  abandoned  in  favor  of  one  that  is 
compatible  with  the  test  plan.  Suppose  that  the  maximum  stress  failure  rate  is  used  instead 
of  the  sum  to  represent  the  device  failure  rate.  This  can  be  estimated  directly  from  the 
tests.  It  recognizes  that  the  various  stressful  environments  have  their  individual  effects,  but 
they  are  not  additive.  There  is  an  extreme  effect  that  dominates  the  others.  Perhaps  the 
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deterioration  in  system  performance  is  represented  by  a  worst  case  or  cases;  there  may  be 
many  paths  to  those  cases  and  the  level  of  such  deterioration  would  increase  with  the 
continued  application  of  stresses.  The  test  plan  merely  specifies  an  amount  and  variety  of 
stresses. 

This  change  in  requirement  is  certainly  far  less  conservative.  It  may  be  optimistic.  It 
seems  worthwhile  to  take  a  look  at  its  effect. 

I.  The  new  reliability  parameter  is  Rq  =  max(/?2,..., R/).  The  empirical  counterpart 
will  be  the  point  estimator 

=max{^2’^3>--M^4 

and  an  ad  hoc  lower  confidence  bound  can  be  constructed  as  follows.  Take  the  total 
sample  size  for  the  stressed  tests 

k 

No  =  J,ni  (17) 

i=2 

and  the  equivalent  number  of  successes  assuming  the  common  success  probability 

Xo=NoRq.  (18) 

Treat  Zq  ~  Bin(lVo,  Rq)  and  use  the  incomplete  Beta  function  method,  Appendix  B. 

n.  This  method  builds  upon  the  log-linear  analysis  of  the  data  (refined  by  failure 
category,  see  Section  6).  The  test  plan,  when  viewed  as  a  contingency  table,  shows  that, 
on  the  basis  of  the  data  from  tests  that  lead  to  product  acceptance,  there  is  no  reason  to 
claim  that  the  treatment  effects  are  not  all  the  same.  This  supports  a  model  that  says  the 
distributions  in  the  columns  of  Table  3  are  the  same.  In  other  words,  we  have  no  evidence 
that  any  of  the  Ri,  /?2»  ->  different  The  data  can  be  pooled  into  a  single  set  of  140 

observations.  Let  RR  be  the  probability  of  success  and  X  be  the  total  number  of  failures. 
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The  point  estimate  of  RR  is  1  -X/140  and  the  lower  confidence  bound  follows  from  usual 
binomial  theory. 

5.  Numerical  Comparisons. 

Let  us  apply  all  six  methods  to  two  data  sets.  They  appear  in  Table  1.  The  first,  DSl, 
was  used  in  [6]  and  likely  is  fictitious.  It  has  k  =  4.  The  second,  DS2,  is  a  threshold  case  in 
the  template  of  Appendix  A;  it  also  appears  in  Table  3.  The  three  rows  are  sample  sizes, 
failure  counts,  and  empirical  success  rates. 


Table  1.  DataSets 


Environment 

1 

2 

3 

4 

5 

6 

7 

DSl 

Sample  Size 

20 

20 

32 

20 

- 

— 

— 

Failures 

1 

1 

2 

1 

— 

— 

— 

Ri 

.95 

.9375 

.95 

.95 

— 

- 

- 

DS2 

Sample  Size 

50 

5 

5 

20 

20 

20 

20 

Failures 

3 

1 

0 

1 

2 

2 

1 

Ri 

.94 

.80 

1.00 

.95 

.90 

.90 

.95 

The  application  of  the  methods  outlined  produce  the  following  results.  (The  DSl  results 
are  not  exactly  the  same  as  those  in  [6]  because,  using  modem  software,  it  is  not  necessary 
to  interpolate  from  Incomplete  Beta  function  tables.)  The  lower  confidence  level  is  0.95. 


Table! 


Sandia 

Gleser 

NFS 

I 

n 

I 

n 

I 

n 

DSl 

R 

0.9461 

0.9461 

0.9375 

0.9375 

0.9375 

0.9457 

Icb 

0.3411 

0.3681 

0.3852 

0.6186 

0.8684 

0.8891 

DS2 

R 

0.8848 

0.8848 

0.7969 

0.7969 

0.800 

0.9286 

Icb 

0.0686 

0.1582 

0.1753 

0.1478 

0.718 

0.8819 

Referring  to  DSl,  the  reliability  estimates  are  generally  high,  but  there  is  considerable 
dispersion  in  the  Icb’s.  The  Gleser  techniques  appear  noticeably  better  than  the  Sandia 
ones.  Although  the  NFS  Icb’s  are  even  better,  do  not  forget  that  they  refer  to  less  stringent 
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definitions  of  the  device  reliability.  Turning  to  DS2,  the  reliability  estimates  are  higher  for 
the  Sandia  technique  than  for  the  Gleser.  None  of  these  two  Icb’s  are  high.  DS2  may  be 
special  in  that  the  observed  maximal  number  of  failures  in  the  unstressed  samples  causes 
the  reliability  to  appear  higher  than  that  for  the  stressed  samples.  It  seems  that  this  has  an 
effect  upon  the  Icb’s  as  well.  In  contrast  to  this,  the  NPS  values  are  attractive.  But  again 
one  must  not  ignore  the  relaxation  of  the  stringency  of  reliability  definition. 

Simple  Modeling. 

Although  the  test  plan  does  not  directly  support  the  requirement  that  the  reliability 
definition  should  include  successful  performance  of  the  device  after  suffering  all  of  the 
stresses,  it  may  contain  enough  information  to  estimate  model  parameters  that  help 
describe  that  goal.  The  following  describes  a  simple  structure  for  accomplishing  this. 
Suppose  that  we  assert  that 

/?2  =  %  =  Rid^,...,Rjc  =  Ridi^  (19) 

where  is  the  probability  that  an  unstressed  device  functions  successfully  and  the  {c?/} 
represent  the  degradation  of  Ri  due  to  the  stress  treatment.  All  d(  must  satisfy  the 
constraint  that  0  <  <  1  so  a  natural  estimator  would  be 

di  =  Tmn{l,Rj/Ri)  (20) 

and  the  estimator  for  the  equation  (1)  definition  of  R  would  be 

R  =  Rifldi.  (21) 

2 

The  estimator  (21)  cannot  be  larger  than  (13).  To  see  this,  use  (20)  in  (21)  and  rewrite 
it  as 

R  =  min(^i ,  ^2 )  ,  ^^3 ). . .  min(^i ,  )/(l  -  (22) 
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and  compare  with  (13).  On  the  other  hand,  the  variance  of  the  numerator  of  (22)  is  smaller 
than  that  of  (13).  Heuristically,  this  should  produce  a  larger  Icb  for  the  reliability  using 
methods  similar  to  Gleser  I  and  H.  Unfortunately  the  sampling  distribution  of  this  statistic 
is  quite  awkward  and  the  lower  confidence  bounds  difficult  to  find. 

The  situation  improves  slightly  if  we  assume  that  all  rf,-  are  the  same,  say  d.  Then  d 
could  be  estimated  with  an  average  of  the  dj  from  equation  (20)  and  the  point  estimator 
would  become 

R  =  Rl\dj  (23) 

but  the  lower  confidence  bound  is  still  difficult 

Let  us  compute  this  point  estimator  for  our  two  examples: 

DSl:  .^  =  .9376  DS2:  .R  =  .8 

In  the  second  case  the  value  is  Ri  because  all  Rj  >  Ri  for  i  =  2,...,7.  The  point  estimates 
are  smaller  but  similar  to  those  of  the  previous  models.  The  advantage  would  appear  in  the 
lower  confidence  bound  once  such  a  computation  has  been  developed. 

6.  Log  Linear  Analysis. 

The  result  of  testing  each  item  in  each  group  results  in  exactly  one  of  the  following 

four  classifications  mentioned  in  Section  2: 

a&e.  Color  display  or  safety  failure 
b.  Ignition  failure 

c&d.  Delay  or  burning  time  specification  failure 
sue.  No  functioning  out  of  zone;  success. 

The  first  three  classifications  are  failure  types  and  the  fourth  identifies  a  successful  test.  So 
far  no  use  has  been  made  of  the  extra  information. 

The  test  plan  specifies  the  sample  size  for  each  group  and  threshold  numbers  for 
accepting  the  lot  by  classification  for  each  group.  Also  there  is  a  dureshold  failure  count 
for  the  entire  sampled  set  regardless  of  the  group  designation.  For  example,  it  may  specify 
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20  signals  for  testing  in  the  transportation  &  vibration  group,  and  allow  no  a&e  failures;  at 
most  one  b  failure;  and  at  most  two  c&d  failures  for  passage  of  the  lot  through  that  group. 
Requirements  of  this  type  are  listed  for  all  the  groups  and,  in  addition,  there  is  a  restriction 
on  the  total  number  of  failures,  by  classification  type,  for  the  entire  sampled  set 

There  are  also  a  number  of  non  destructive  inspections,  but  they  are  of  no  concern  in 
the  development  of  reliability  analysis  for  the  sampling  plan  outlined. 

Analysis, 

It  is  recommended  that  the  test  plan  be  viewed  as  a  two-way  table  of  counts  as 
exemplified  in  Table  3.  (This  is  DS2  with  greater  detail.) 


Table  3 


5ft 

40ft 

tran&vib 

temp&hum 

hi  temp 

lo  temp 

no  stress 

total 

a&e 

0 

0 

0 

0 

0 

0 

0 

0 

b 

0 

0 

1 

1 

•0 

0 

1 

3 

c&d 

1 

0 

0 

1 

2 

1 

2 

7 

sue 

4 

5 

19 

18 

18 

19 

47 

130 

total 

5 

5 

20 

20 

20 

20 

50 

140 

Each  column  of  this  table  is  the  result  of  multinomial  sampling  from  a  four-cell 
classification  and  the  column  total  marks  the  sample  size.  The  columns  are  statistically 
independent  and  are  identified  with  the  stress  groups.  Let  n{i,  j)  be  the  counts  of  items 
falling  into  the  classification  in  the  stress  group,  and  let  p(i,j)  be  the  multinomial 
probabilities  for  column y,  i  =  1,...,4.  The  maximum  likelihood  estimator  for  the  reliability 

of  the/*’  group  is  n(4, /)/«(.,/)  where  n(.,j)  is  the  sample  size  for  the Z*’  group.  These  are 
the  same  /?,■  numbers  used  earlier. 

Let  us  fit  a  log-linear  model,  ref  [1,4,5],  to  our  two-way  table.  The  general  plan  is  to 
use  the  estimated  cell  probabilities  from  an  acceptably  fitted  model.  The  minimum  of  the 
column  reliabilities  has  been  proposed  for  the  estimated  lot  reliability.  However  the  test 
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plan  is  such  that  the  distinctions  between  the  various  stresses  are  not  significantly  different 
in  the  case  of  acceptable  lots.  (But  this  could  change  if  the  sample  numbers  were  larger.) 
The  common  success  probability  is  more  readily  estimated  as  done  in  NFS  11.  The  log- 
linear  model  approach  is  described  first 

Let  be  the  expected  value  of  log{ «(/,/)}.  The  two-way  layout  model  has  been 

acceptably  fitted  for  the  test  plans  described  above.  In  algebraic  notation  it  is  described  by 

m{i,  y )  =  jLt  +  /if  -t-  /ly  for  i  =  1. .  .4  and  y  =  1. .  .7  (24) 

and  linear  contrast  constraints  [1,4]  on  the  |^f }  aiid  \^j\  ^PP^y- 

The  maximum  likelihood  estimates  are  easily  found  (either  algebraically  or  with  commonly 
available  statistical  software).  Indeed  the  cell  probabilities  are  the  classical  contingency 
table  estimators.  When  the  above  log-linear  model  is  tested  against  the  saturated  model, 
the  fit  cannot  be  rejected  for  data  that  are  accepted  by  the  test  plans.  Hence  it  follows  that 
this  procedure  is  the  classical  test  for  a  common  (multinomial)  column  distribution  for  the 
four  classified  outcomes.  This  in  turn  justifies  the  pooling  of  the  column  data  into  a  single 
four-celled  multinomial  distribution  and  usmg  the  empirical  probability  of  the  fourth  row 
as  the  estimated  reliability.  For  the  example  above  this  value  is 

«=‘“=.929. 

140 

The  95%  lower  confidence  limit  is  .8819  (from  usual  binomial  formulas).  Finally,  keep  in 
mind  that  this  example  was  chosen  at  the  boundary  threshold  of  acceptability  according  to 
the  test  plan  rules. 

The  model  (24)  could  have  included  some  interaction  terms  of  the  form 

m 

A  more  detailed  appropriate  model  could  be  developed.  It  would  require  more  data  in 
order  to  estimate  its  terms.  Once  done  one  can  compute  probabilities  of  the  form 
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Pjii)  =  Pr{  outcome  i  given  stress  j  has  been  applied}  =  exp| /t + A-f  +  Ay  +  A^  j .  (25) 

A  model  of  this  type  could  be  used  to  allocate  the  total  sample  over  the  various  stress 
types  and  to  specify  the  decision  rule  for  each  outcome  category.  One  objective  would  be 
to  avoid  the  condition  of  a  small  sample  size  for  a  stress  type  having  undue  influence  on 
the  estimates  of  system  reliability.  (Such  has  occurred  for  the  five  foot  drop.) 

It  appears  that  new  decision  rules  are  being  proposed;  they  do  not  allow  any  failures  in 
order  for  a  lot  to  be  acceptable.  It  seems  to  this  author  that  such  a  rule  could  produce 
considerable  waste.  The  Operating  Characteristic  function  is  of  the  form 

P"  (26) 

where  p  is  the  probability  of  a  successful  test  and  n  is  the  number  of  items  tested.  Suppose 
Pi  is  the  lower  limit  specified  for  success.  Values  of  pi  around  .95  and  sample  sizes  of 
about  100  produce  rather  large  Type  I  error  probabilities.  That  is,  many  acceptable  lots 
will  be  discarded. 

It  seems  wiser  to  do  more  testing,  keep  better  records,  and  develop  sharper  decision 
rules  based  upon  careful  modeling  such  as  that  suggested  above.  Then  the  process  would 
be  better  understood  and  far  fewer  acceptable  lots  would  be  thrown  out. 

7.  Summary. 

The  numerical  comparisons  in  Table  2  for  Sandia  and  Gleser  serve  to  illustrate  the 
conservative  nature  of  the  methods  in  place.  The  Gleser  n  method  showed  improvement 
for  DSl,  but  seemed  to  break  down  when  used  with  the  DS2  data.  The  NFS  approach 
shows  attractive  numbers  but  the  interpretations  are  not  the  same. 

The  author  argues  that  the  test  plan  cannot  support  the  original  goal;  at  least  not 
without  some  additional  modeling.  To  begin  with,  no  sample  is  subjected  to  aU  of  the 
stresses  and  hence  there  is  no  direct  testing  of  this  condition.  Further,  the  requirement 
does  not  specify  the  order  of  application  of  the  various  stresses.  For  example,  suppose  that 


17 


the  vibration  stresses  are  applied  first.  They  hasten  the  loosening  of  the  seals.  This  in  turn 
makes  the  sample  more  susceptible  to  the  effects  of  the  temperature  and  humidity  stresses. 
On  the  other  hand,  suppose  that  the  vibration  stresses  are  applied  after  the  temperature 
and  humidity  stresses.  Then  the  original  resistance  to  temperature  and  humidity  is 
maintained.  Other  scenarios  reflecting  the  importance  of  order  can  be  concocted. 

The  simple  modeling  in  Section  5  may  offer  improverr^nt,  but  it  does  nothing  to  offset 
the  comments  of  the  previous  paragraph.  The  log  linear  analysis  uses  more  of  the  test 
result  information.  If  pursued,  the  results  could  be  valuable  for  use  in  the  design  of  test 
plans.  The  nature  of  the  test  plan  can  have  substantial  effects  on  the  estimation  of 
reliability.  Recall  that  a  single  failure  in  the  5  ft  drop  stress  can  have  a  maiked  effect  on  the 
reliability  estimate  simply  because  of  the  low  sample  size. 

Unfortunately  a  large  amount  of  data  must  be  analyzed  in  order  to  pick  up  any 
interaction  information.  Also  there  is  likely  to  be  a  manufacturer  effect  that  should  be 
considered.  Another  idea  would  be  to  apply  multiple  stresses  to  the  individual  items.  The 
log  linear  analysis  could  be  quite  useful  if  it  were  decided  to  do  this.  Perhaps  simply 
applying  them  in  pairs  could  reveal  useful  information. 

At  the  present  there  appears  to  be  no  information  relating  the  effect  of  sustaining  a 
stress  over  a  variety  of  time  intervals,  and  not  just  those  specified  [12].  What  is  needed 
further  is  some  information  that  can  allow  the  modeling  of  the  failure  rates  with  time  and 
how  the  various  stresses  compress  time. 
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APPENDK  A 

Excerpts  from  Test  Plan  WS13697 

The  following  table  summarizes  the  decision  rules  for  the  destructive  testing  portion  of 
the  cited  inspection  plans,  Table  I  (plan  A).  The  notation  k/{k  +  1)  means:  ‘accept  with  k 
or  fewer  failures/reject  with  ^  +  1  or  more  failures’.  The  failure  outcomes  are  as  in 
Section  2. 


Stress  Type 

sample  size 

outcome 

rule 

i  Five  foot  drop 

5 

a&e 

0/1 

b,c&d 

1/2 

ii  Forty  foot  drop 

5 

a&e 

0/1 

b/c&d 

1/2 

iii.  Vibration 

20 

a&e 

0/1 

b 

1/2 

c&d 

2/3 

iv.  Temperature  &  Humidity 

20 

a&e 

0/1 

b 

1/2 

c&d 

2/3 

V.  High  Temperature 

20 

a&e 

0/1 

b 

1/2 

c&d 

2/3 

vi  Low  Temperature 

20 

a&e 

0/1 

b 

1/2 

c&d 

2/3 

vii  No  Stress 

50 

a&e 

0/1 

b 

1/2 

c&d 

3/4 

total 

140 

Group  requirement  (for  the  above  tests) 

140 

b 

3/4 

c&d 

7/8 

The  group  requirement  places  an  additional  limitation  on  the  total  acceptable  number  of 
failures,  by  failure  outcome.  (Of  course  no  a&e  failures  are  tolerated.) 
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APPEP^IXB 

The  Use  of  the  Incomplete  Beta  Function  to  Produce  Binomial  Confidence  Limits 


Suppose  X  is  a  binomial  (n,  p)  random  variable.  The  cumulative  distribution  function  is 
related  to  the  incomplete  beta  function  by  the  well  known  formula 


P(a:)  =  P{X<x}  = 


=  l-J&{p,x+\,n-x) 

and  the  incomplete  beta  function  is  given  by 

The  Maximus  [9]  methodology  employs  a  technique  of  identifying  subsystems  of 
components  with  a  fictional  equivalent  component.  The  reliability  of  the  subsystem  is 
estimated  from  the  results  of  testing  its  components  and  the  result  is  viewed  as  a  binomial 
proportion.  But  the  parameters  of  this  equivalent  component  need  not  be  integral 
numbers.  That  is,  neither  the  sample  size  nor  the  number  of  successes  need  be  whole 
numbers.  This  leads  to  the  need  for  some  interpolation  formulae.  Incomplete  Beta  function 
software  is  readily  available,  so  one  may  as  well  use  it.  The  coefficient  in  IB  is  related  to 
the  Beta  function 


~ [Beta(j:, n-x-l)]  ^ 

and  there  is  no  longer  any  requirement  that  ;c  and  n  be  integers. 

To  find  a  1  -  a  level  lower  confidence  limit  for  p  one  seeks  the  solution  of 

=  '^{p,x,n  +  l-x) 

as  a  function  of  p.  When  :t  =  0,  use  zero  for  the  Icb. 
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To  find  a  1  -  a  level  upper  confidence  limit  for  p  one  seeks  the  solution  of 


a  = 


When  x  =  n,  use  unity  for  the  ucb.  Software  for  computing  the  inverse  incomplete  beta 
function  is  available.  For  example,  the  function  qbeta(p,x+l,  n-x)  in  the  SPLITS 
package  [13]  is  used  in  our  work. 
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APPENDIX  C 

The  Variance  of  a  Product  of  Independent  Random  Variables 


Let  fif  =  E(X{)  and  (jf  =  V(Xj),  for  j  =  1, 2, ... .  The  formula  for  the  variance  of  the 
product  is: 

4nx/l=4nx?|-n(M?) 

u=i  J  1 1  J  1 

1 

From  this  one  can  verbalize  the  expanded  series  in  the  following  way.  It  is  a  senes  of 
terms  having  k  factors  per  term.  Each  factor  is  of  second  order,  i.e.  either  a  or  a  6^ 
(with  subscripts).  Every  combination  of  and  must  appear,  there  is  a  term  consisting 
of  the  product  of  all  variances;  there  is  no  term  which  is  the  product  of  all  squared  means, 
so  2^  -  1  terms  in  total. 

For  our  application  theXj  are  binomial  proportions  so 

^i=Pi  and  aj  =pi{l-pi)/ni  foTi  =  l,...k. 

If  the  {n,  }  are  large  then  we  can  ignore  aU  terms  that  have  two  or  more  of  the  .  This 

leads  to  the  simplified  approximation 


V{hP2-Pk)  =  ^ 


^  P/(l  Pi) 


UpJ 


PlP2  •  •  •  Pi-lPi+1  ■••PkO-- Pi) 

rii 


(Cl) 


All  of  the  ignored  terms  are  positive. 
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APPENDIX  D 
Details  for  a  Delta  Method 


We  develop  the  delta  method  for  approximating  the  variance  of 


W  =  X/F  (D.l) 

assuming  X  and  Y  are  independent  random  variables  and  r  is  an  integral  constant 
Start  with  the  first  order  Taylor  expansion  of  W(X,  Y)  about  (xq’  iVo)- 

W  =  Xo/y^+(X-xo)/y^-  rxQ  (7  -  yo )/  • 

The  remainder  terms  are  assumed  negligible  when  (xq,  Jq)  is  the  centroid  of  the  (X,  Y) 
distribution.  Then 


Var(W) «  Var(X)/y^’'+{x^r^/yi’"^^)Var(Y), 
The  present  application  calls  for  X  =Rp,  7  =  (l  —  and  r  =  k-2.  Then 


(D.2) 


(D.3) 
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APPENDIX  E 

Alteration  of  Empirical  Proportion  Estimates 

When  X  is  distributed  ~  Bin(/i,  p)  the  maximum  likelihood  estimator  for  p  is  the 
empirical  proportion  of  successes.  This  produces  values  of  zero  or  one  in  the  extreme 
cases  of  X  =  0  or  X  =  n,  respectively.  In  many  applications  such  values  are  either 
inappropriate  or  unusable  and  relief  from  these  extremes  is  required.  A  popular  form  of 
relief  is  to  use  one  sided  confidence  limits  as  point  estimates  in  these  two  cases.  In  the  case 
X  =  n  the  1  —  05  lower  confidence  limit  for  p  is  computed  from  the  equation 

p'^  =  a. 

and  in  the  case  X  =  0  the  1  -  a  upper  confidence  limit  is  computed  from 

(l-p)«  =  a. 

The  Maximus  technology  [9]  uses  a  =  0.7.  The  following  graphs  provide  some  visual 
comparisons  of  estimators  of  this  type  by  plotting  the  estimators  for  several  choices  of  a 
as  a  function  of  n.  Also  included  are  some  maximum  likelihood  estimate  curves  for  values 
of  X  ‘nearby’.  The  ‘X  =  0’  curves  are  simply  one  minus  the  ‘X  =  rC  curves. 


High  X  Proportion  Estimates 
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